Lossy Video Coding

We propose the concept of Spatio-Temporal Model-Assisted Compatible (STMAC) coding and Motion Adaptive Spatio-Temporal Model-Assisted Compatible (MA-STMAC) coding, two techniques that selectively encode areas of diierent importance to the human eye in terms of space and time in moving images. Our motivation is the fact that eye contact and lip synchronization are very important in person-to-person communication. Several areas including the eyes and lips require diierent types of quality since they have diierent perceptual signiicance to human observers. For example, for the eyes high resolution is needed for clear communication , while for the lips frequent refresh is desirable. Fully automatic algorithms are described for performing model area detection and motion tracking. An Area Selective (AS) rate control scheme is proposed, in order to eeect area-selective bit allocation. The decoder does not need to be changed in any way although the encoder's rate control unit is modiied. As a result, the scheme can be used in compliance with standards-based codecs. As the scene model is only used in the rate control module (hence the term \model-assisted"), the technique does not suuer from the artifacts associated with model-based coding. Extensive experimental results are provided using ITU-T H.263, addressing very low bit rate compression (below 64 Kbps).